There is a great interest in surgical and other arenas to provide for the real-time computerized measurement of a spatial position and orientation (pose) of specially marked objects moving unpredictably through an unstructured scene. For one example application, the scene may be an operating room in which surgery is performed, and the tracked objects are surgical instruments, and implants to be secured to the patient's anatomy in pre-planned positions. The pose of the surgical instruments is monitored by position sensors, such that the real-time location of the instruments can be displayed on previously acquired patient image data. The pose of the patient is also typically tracked separately, to allow for sensing of anatomy movement relative to the tracked instruments during the operation. In another example application, the scene is a movie set in which the tracked objects are the bodies of actors for the purpose of replacing them with computer-animated substitutes performing the same motions.
In such applications, the various poses of the tracked objects need to be determined within a typical accuracy of 1:1,000 of the total volume of measurement in each spatial direction, i.e. 1 mm within 1 metre cubed. To date, optical pose tracking systems are considered the most reliable for determining the pose of tracked objects to selected accuracies, and are therefore used in the majority of applications for tracking objects through unstructured scenes. Optical pose tracking systems are commercially available, for example, from Northern Digital Inc. of Waterloo, Ontario, Canada. These systems use two or more optical sensors to optically detect the pose of the tracked objects. Examples of optical systems can be found in U.S. Pat. No. 6,351,661 by Cosman and U.S. Pat. No. 6,351,659 by Vilsmeier, the contents of which are herein incorporated by reference.
Optical pose tracking systems differentiate between tracked and non-tracked objects in the scene by placement of special markers, or “targets”, on the objects. A stationary stereoscopic optical sensor, containing two cameras, then repeatedly measures the position and, optionally, the orientation of the individual targets relative to the sensor's own spatial frame of reference, and then reports the measured pose of the targets to a computer for further processing. The computer processor uses the measured pose from the sensor, and prior knowledge of the spatial relationship between the targets and defined points on the objects to which they are attached to calculate the pose of each such object relative to other defined objects or reference frames. However, one disadvantage with current optical tracking systems is that differentiation between the markers and the scene background can be hindered by inadequate illumination of the markers.
In current optical tracking systems, the light energy associated with the targets and sensed by the optical sensors may either be: generated at the target, referred to as active targets; generated near the sensor and reflected by targets coated with a special retro-reflective material, referred to as actively illuminated targets; or an ambient light energy, referred to as fully passive targets. A survey of the types of known optical targets and associated tracking systems appears in the article “An analysis of the properties of targets used in digital close range photogrammetric measurement”, T. A. Clarke, SPIE vol. 2350 (1994) p. 251–262, herein incorporated by reference. The article discusses that commercially available pose tracking systems almost universally use either: infra-red LEDs emitters (IREDs) as active targets; or spheres or circles coated with IR reflective 3M™ Scotchlite™ as actively illuminated targets, which are illuminated by infra-red radiation projected from IREDs placed near the sensors. One disadvantage of such systems is that they identify targets by relying upon a strong illumination contrast present between the targets and the background, which is artificially created by the emitted or reflected infra-red radiation. However, the magnitude of the illumination contrast can be affected by a malfunction in the operation of the IREDs and/or obstructions positioned between the optical sensors and the emitted or reflected infra-red radiation.
Alternatively, fully passive systems have a number of inherent advantages over active systems since, by not employing an active infra-red energy source, their optical sensor and target configurations as compared to the active systems have the potential to be simpler, smaller, lighter, less expensive, more reliable, and not dependent upon the projection range of the energy source. Unlike active systems, however, the brightness of targets relative to their surrounding in passive systems cannot be controlled, which can make it difficult to perform the tasks of (1) reliably detecting the presence of targets due only to increased reflection intensity, (2) uniquely labeling each target, and (3) pinpointing each target's pose to a sub-millimeter accuracy over a large measurement volume. Known designs of fully passive systems are such as those described in U.S. Pat. No. 5,792,147, associated UK patent number 2,246,261, and in U.S. Pat. No. 5,603,318, which are herein incorporated by reference. However, these known passive systems have not provided an effective solution to all the performed tasks, and therefore have not been commercially successful.
In particular, U.S. Pat. No. 5,792,147 by Evans at al. describes a passive pointer with a marked predetermined pattern thereon to identify the tip of the pointer in relation to the patient's body, which is then displayed on a monitor with the pre-operative and intra-operative patient images. The border of the pattern in relation to the background is identified using the contrasted edge between the area internal to the target versus the surrounding background area. The predefined patterns are composed of a series of dark-light edges, which are identifiable by an image processing algorithm. Evans et al. describe the use of two high contrast patterns, which are placed in proximity to one another on markers to form the target. The first pattern comprises a complex shape that is not commonly encountered in the selected scene. This recognizable pattern when applied to the marker is used to help uniquely identify and distinguish the target from its surroundings by pattern recognition techniques, on video images supplied by the optical sensors. The second pattern comprises a few straight edges, which are used for incrementally tracking and pinpointing the target's pose.
However, the described patterns and their use in the optical tracking system by Evans et al. have two major limitations. The first limitation is that the image processing operations required to identify the presence of the complex shape, at a wide range of orientations and visible sizes on the video image, are complex and require a substantial amount of CPU overhead. This excessive computation can hinder the application of the targets for real time object tracking. Further, the target must be held stationery to facilitate the pattern recognition, a situation not always desirable in applications where objects move unpredictably through unstructured scenes. The second limitation is that the method by Evans et al. allows for the incremental tracking of the target's pose, and relies on the target remaining continuously and fully visible while moving a distance of only a few pixels between subsequent measurements. Accordingly, the combination of these limitations results in a termination of pose measurement whenever the target is moved rapidly or when the line of sight between one of the sensor cameras and the target is interrupted. Thereafter, resumption of measurement requires the user to reposition the target in a known location for some period of time to allow the target to be reacquired for subsequent tracking. Other drawbacks of the system by Evans et al. include the need to perform a calibration sequence prior to each measurement session, and a lack of sufficient measurement accuracy for small targets, which require the targets to be relatively large with respect to other objects in the background of the video images supplied by the optical sensors.
Furthermore, in U.S. Pat. No. 5,603,318, Heilbrun at al. disclose a fully passive system in which a complex calibration task is performed prior to each measurement session, similar to Evans et al. Following the calibration task, uniformly colored spherical targets are detected with the aid of background subtraction and edge detection. However, one disadvantage with uniform spherical targets is that they have a widely varying appearance in video images under different lighting conditions, and therefore a static reference background illumination cannot be guaranteed in many operating environments. A further disadvantage is that a sufficient color or intensity contrast may not always be present between each target and its background, e.g. the contour of a white sphere over a white background cannot be distinguished. As a result, the operation of the system by Heilbrun et al. can be inconvenient, unreliable, and inaccurate for target configurations in certain scenes.
It is an object of the present invention to provide an optical pose tracking system and method to obviate or mitigate at least some of the above presented disadvantages.